MPEG-4 Synthetic Video in real implementation

MPEG-4 is the international coding standard developed for information transfer via low bit-rate communication channels. This article is the report on an experimental implementation of full-automatic pipeline MPEG-4 Synthetic Video Facial Animation (Simple Profile). This REAL-TIME pipeline includes the automatic detection, encoding, network transfer, and decoding of the facial animation parameters (FAPs) as well as decoder proprietary face model animation and rendering. To obtain the animation parameters the authors developed the methods of automatic facial region detection combined with the recognition and tracking of the feature points for eyes, eyebrows and mouth (the inputs for further FAPs calculation). For that the algorithms of adaptive color segmentation, correlation analysis, optical flow and multilevel vector analysis were implemented. Model-independent 3D face model animation method based on the FAPs’ regions of influence detection upheld by the feature points information was used. Rendering based on OpenGL is synchronized with audio stream and includes eyelashes and “expression wrinkles” modeling. The animation results detailed in the article are based on realistic video data input sequences.